Robotics Technology 

Perception, User Interfaces and Architecture 







Terry Fong 

Intelligent Robotics Group 
NASA Ames Research Center 
terry.fong@nasa.gov 


NASA Ames Intelligent Robotics Group 


Overview 

• 36 researchers (17 Ph.D.’s) 

• 25+ student interns yearly 

• 80% NASA work 

• 20% non-NASA work 

• SBIR-STTR (Phase 1,2, 2E, & 3) 


Research themes 

• Automated planetary mapping 

■ Base maps & terrain models 

■ Geospatial data systems 

• Exploration user interfaces 

■ Robot & science operations 

■ Accessible science data 

• Robots for human explorers 

■ Improve efficiency & productivity 

■ Free-flyers, lake lander, & rovers 




irg.arc.nasa.gov 
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IRG Collaborations (2014-2015) 


Academic 



M Case Wes tern Reserve 

Ifcll UNIVERSITY EST l8i6 


mii 

Berkeley Rutgers 

UNIVERSITY OF CALIFORNIA V ofnewjersey 


University of Idaho 




WISCONSIN 



UNIVERSITEIT 

GENT 



SETI Institute 


KAI5T 

Korea Advanced Institute of Science and Technology 



Intelligent Robotics Group 


Commercial 
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Robotics for Human Exploration 


Purpose 

• Increase human productivity 

• Improve mission planning & execution 

• Transfer some tasks to robots 
(tedious, repetitive, long-duration) 

Before Crew 

• Recon (scouting) & prospecting 

• Site prep, deploy equipment, etc. 

Supporting Crew 

• Inspection, mobile camera, etc. 

• Heavy transport & mobility 


After Crew 


• Follow-up & close-out work 

• Site survey, supplementary tasks, etc. 
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Robots 
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User Interfaces 
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Virtual Exploration (VERVE) 
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Exploration Ground Data System (xGDS) 
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Software Architecture 



Strategic Data Producers 


Tactical Data Producers 
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xGDS 


Mission Management Team 
Tactical Science Ops 
Strategic Science Ops 


xGDS Server Tools 
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K10 Robot at Haughton Crater, Canada 
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K10 Remotely Operated from ISS 


Intelligent Robotics Group 


11 




Intelligent Robotics Group 


12 



Mojave Volatiles Prospector 



Intelligent Robotics Group 


13 




NASA-Nissan Partnership 


R&D agreement (5 year term) 

• Autonomous vehicle systems 

• Human-machine interface 

• Network-enabled applications 

• Software analysis and verification 

• Vehicle testing at NASA Ames 

Robotics technologies 

• Application of NASA algorithms and concepts 

• Joint development, testing, and assessment 

• Demonstration: fleet management of multiple autonomous vehicles 







Mars in Google Eart 


Explore Mars in 3D 

• Released Feb. 2, 2009 

• Co-developed with Google 

• NASA Ames created content 
& processing scripts 

Content 

• Global maps: topography, 
infrared, historical, etc. 

• Imager footprints & overlays 
(HiRISE, CTX, MOC, ...) 

• Mars rover tracks & 
color panoramas 

• Tours (Bill Nye & Ira Flatow) 

• Live from Mars: THEMIS 

• And much more ... 
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Moon in Google Earl 


Explore the Moon in 3D 

• Released July 20, 2009 

• Co-developed with Google 

• NASA Ames created content 
& processing scripts 

Content 

• Global maps: topography, 
geologic, historical, etc. 

• Spacecraft imagery: Apollo, ( 
Lunar Orbiter, etc. 

• 3D models of spacecraft, 
landers, and crew rovers. 

• Tours (Andy Chaikin, Buzz 
Aldrin & Jack Schmidt) 

• And much more ... 
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World WideT elescope 


Complete HiRISE Mosaic 

• Mars Reconaissance Orbiter 
HiRISE imager 

• 74,000 images 

• Each image: 20K x 50K pixels 
(> 1 GB / image) 


Mosaic stats 


Tile Dimensions 

256 x 256 pixels 

Root Tiles / Image 

15,000 

Tile Space 

25 KB 

Tiles Total 

229 million 

Total Mosaic Size 

5.7 TB 
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Worldwide Telescope 


Welcome to WWT | Mars 
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Ames Stereo Pipeline ... for Earth 



3D terrain modeling 

• Digital Elevation Models (DEM) 
produced from satellite images 

• 60% success rate processing 
Digital Globe stereo pairs 
without human input 

• Available as open-source 
(Apache 2 license) 

• Binaries for Linux & OS-X 

Earth science studies 

• Glacier volume changes and 
movement 

• Evolution of subglacial lakes 

• Sea ice movements 

• Climate change measurements 

irg.arc.nasa.gov/ngt/stereo 
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Jakobshavn Glacier DEM 
(2500 sq. km @ 3 m/pix) 




Open Source Software 


Vision 

Workbench 


Apache 2 


RoverSW 
NOS A 1.3 



Neo Geography 
Toolkit 

(Ames Stereo Pipeline) 


Apache 2 




NASA Tensegrity 
Robotics Toolkit 


Apache 2 
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Exploration Ground 
Data Systems 
(xGDS) 


Apache 2 



Visual Environment 
for Remote Virtual 
Exploration (VERVE) 


Apache 2 



RAPID 

(NASA robot 
middleware) 


Apache 2 
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Questions? 



Intelligent Robotics Group 

Intelligent Systems Division 
NASA Ames Research Center 

irg.arc.nasa.gov 
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